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Abstract 

The  Warrior  Preparation  Center  (WPC)  at  Einsiedlerhof  AS,  Germany  provides  JTF  staff- 
level  computer  training.  During  a  recent  mission  rehearsal,  WPC  was  asked  to  conduct  an  air 
campaign  analysis.  As  a  result  of  this  analysis,  WPC  is  looking  at  ways  to  improve  its  current 
capabilities.  Since  there  is  limited  data  published  on  this  topic,  investigation  of  the  air  campaign 
analysis  capabilities  at  key  Air  Force  (AF)  sites  was  the  primary  source  of  data.  The  key  AF 
sites  included  in  this  study  were  WPC,  Checkmate,  the  Command  &  Control  Training  and 
Innovation  Center  (C2TIC),  the  Air  Force  Wargaming  Institute  (AFWI),  and  the  Air  Force 
Agency  for  Modeling  and  Simulation  (AFAMS).  The  research  focused  specifically  on  air 
campaign  analysis  processes  and  tools.  The  findings  reveal  common  themes,  innovative 
processes,  and  several  candidate  analysis  tools.  An  analysis  of  the  findings  led  to  the 
development  of  a  notional  air  campaign  analysis  process  and  a  comparison  of  the  existing 
analysis  tools.  Subsequently,  the  notional  air  campaign  analysis  process  and  best  tools  were 
combined  into  real-world  applicable  air  campaign  analysis  process.  Finally,  an  improvement 
game  plan  was  recommended,  including  implementation  considerations  and  potential  areas  for 


vii 


future  research. 


Chapter  1 


Introduction 

The  application  of  air  power  is  profound  both  in  its  effects  and  its  potential  for  loss  of 
human  life.  It  is  imperative  that  the  warfighter  is  given  every  opportunity  to  apply  air  power 
with  maximum  effects  against  our  adversaries  while  keeping  friendly  losses  to  a  minimum.  The 
actual  application  of  air  power  is  achieved  through  the  execution  of  the  final  product  from  air 
campaign  planning,  the  Air  Tasking  Order  (ATO).  It  is  essential  that  when  an  ATO  is  carried 
out,  the  air  campaign  is  well  planned,  fully  analyzed,  and  thoroughly  rehearsed  before  a  single 
aircraft  takes  flight.  An  air  campaign  planning  process  exists  today,  but  is  not  standardized, 
often  slow,  and  lacks  tools  and  processes  to  analyze  the  quality  of  the  air  campaign  plan,  or 
ultimately,  the  ATO.  Not  surprisingly,  several  Air  Force  (AF)  organizations  currently  conduct 
analysis  for  certain  portions  of  the  air  campaign  planning  process.  A  few  conduct  rehearsals  of 
the  ATO  itself  via  Modeling  and  Simulation  (M&S)  since  rehearsals  with  actual  aircraft  is 
infeasible.  This  research  project  examines  one  particular  AF  organization  that  does  both,  the 
Warrior  Preparation  Center  (WPC),  and  has  subsequently  asked  for  help  to  improve  its  current 
capability. 

More  succinctly,  the  goal  of  this  research  is  to  recommend  ways  for  WPC  to  improve  its  air 
campaign  analysis  capability.  To  do  so,  this  paper  will  first  lay  the  foundation  of  the  research. 
Subsequent  chapters  will  present  its  findings,  analysis  of  those  findings,  and  then  provide 
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conclusions  from  the  analysis.  To  adequately  lay  the  foundation  for  this  research,  this  chapter 
will  provide  relevant  background  information  and  the  statement  of  the  problem,  the  research 
methodology  used,  and  the  associated  limitations  and  assumptions. 

Background  and  Problem  Statement 

The  WPC,  traditionally  a  training  organization,  recently  conducted  its  first  significant  air 

campaign  analysis  and  rehearsal  process.  The  exercise  was  called  Nimble  Lion,  and  the  Initial 

After  Action  Review  briefing  concluded:  1)  the  need  to  “Grow  ability  to  assess  and  analyze 

campaigns,”  2)  “no  robust  analysis  tools  available”  as  a  LIMP  AC,  and,  3)  “Rehearsal  is  the  core 

of  the  analytical  process.”'  Subsequent  queries  to  WPC  regarding  potential  research  topics  of 

interest,  resulted  in  the  following  request: 

I  have  an  ACSC  project  for  you.  We  are  using  our  stochastic  simulations  in  an 
analytical  role.  Essentially,  our  models  allow  us  to  perform  mission  rehearsals  on 
ATOs,  doing  a  fly-out  for  ATO  visualization,  analysis  and  feedback  into 
modifying  and  improving  the  ATO,  and  then  overall  campaign  analysis.  There’s 
a  lot  of  questions  on  how  to  do  this  better,  and  we  are  working  projects  to 
implement  improvements.^ 

A  final  testimony  to  WPC’s  desire  to  grow  their  mission  rehearsal  capability  is  their  recent  four¬ 
fold  manpower  increase  in  their  analysis  directorate  and  subsequent  changes  to  their  WPC 
mission.  The  latest  WPC  mission  briefing  now  includes  mission  rehearsal  as  one  of  their  four 
core  competencies  and  it’s  their  first  bullet  on  their  WPC  After  Next  slide.  Incidentally,  initial 
exploration  revealed  this  in  not  just  a  WPC  problem,  but  actually  an  AF-wide  concern.  Actually, 
the  Air  Force  Agency  for  Modeling  and  Simulation  (AFAMS)  has  been  chartered  to  develop  a 
roadmap  to  improve  M&S  decision  support  tools  for  the  warfighter.  Given  the  potentially  broad 
scope  of  this  research  topic,  the  project  focuses  on  improvements  to  WPC’s  air  campaign 
analysis  tools  and  processes.  The  methodology  is  discussed  in  the  next  section. 
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Research  Methodology 


The  following  section  will  outline  the  fundamental  research  approach,  discuss  data  sources 
and  collection,  and  the  analysis  methodology.  Preliminary  investigations  showed  severely 
limited  information  published  on  this  topic.  So  a  research  methodology  that  didn’t  rely  on  the 
more  traditional  method  of  library  data  collection  was  required.  Previous  experience  indicated 
several  sites  that  also  analyzed  air  campaigns.  By  investigating  their  processes  and  tools,  some 
insights  useful  to  improving  WPC’s  air  campaign  analysis  capabilities  were  discovered.  At  the 
onset  of  the  research,  it  was  believed  that  at  the  worst,  the  result  would  be  a  compendium  of  tools 
and  processes  without  any  meaningful  insights.  This  alone  is  more  information  than  is  currently 
published.  Fortunately,  the  findings  and  subsequent  analysis  provide  a  synergistic  coupling  of 
innovative  analysis  process  ideas  with  optimum  analysis  tools.  Based  on  previous  knowledge 
and  discussions  with  CADRE,  the  sites  of  interest  were  narrowed  down  to  AF  sites  that  were 
“end-users”  of  M&S  analysis  tools.  They  were  further  narrowed  to  a  list  of  sites  that  could  be 
observed  in  minimal  time  and  least  cost.  This  resulted  in  the  following  sites:  the  WPC  itself  (as 
a  baseline).  Checkmate,  the  Command  and  Control  Training  and  Innovation  Center  (C2TIC)  and 
the  Air  Force  Wargaming  Institute  (AFWI).  AFAMS  was  subsequently  added,  although  not  an 
“end-user,”  because  of  their  on-going  study  efforts  of  developing  a  roadmap  for  improving 
decision  support  tools  for  the  warfighter.  Interviews  were  selected  over  surveys  as  the  primary 
data  collection  method  based  on  reading  an  Air  Force  Institute  of  Technology  research  methods 
textbook."^  The  interview  procedure  itself  was  designed  based  on  this  text.  Follow-on 
discussions  via  email  were  anticipated,  and  thankfully  occurred.  After  collecting  the  data  from 
the  sites,  a  qualitative  analysis  of  the  air  campaign  analysis  processes  and  tools  was  conducted. 
Next,  a  coupling  of  the  process  improvements  and  ideal  tools  were  established.  Finally,  an 
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implementation  strategy  to  incrementally  improve  WPC’s  air  campaign  analysis  capability  was 


developed. 


Limitations  and  Assumptions 

It  is  worth  noting  some  of  the  limitations  and  assumptions  regarding  this  research  project.  A 
substantial  limitation  is  the  lack  of  published  data  on  the  topic.  Another  limitation  is  the 
relatively  short  time  period  to  conduct  the  research,  the  abbreviated  nature  of  the  final  product, 
and  scare  resources.  Finally,  the  inherently  human  involvement  in  the  air  campaign  analysis 
process  precludes  quantitative  analysis  of  the  findings.  These  subsequently  limit  the  scope  of 
this  research  in  the  following  ways: 

•  Data  collected  from  the  key  AF  sites  is  critically  important  to  the  research  findings 

•  The  number  of  analysis  processes  and  tools  observed  are  not  all-inclusive 

•  An  analysis  of  the  tools  relies  heavily  on  the  interviewee  and  documentation  available 

•  Air  campaign  analysis  and  tools  focus  on  operational  modeling,  excluding  logistics 

modeling 

•  Analysis  is  highly  qualitative  in  nature,  excluding  rigorous  performance  benchmarking 

and  quantitative  tool  life  cycle  cost  analysis 

It  should  be  noted,  that  these  limitations  did  not  seriously  hinder  the  research  and  were  mitigated 
whenever  possible.  There  are  three  major  assumptions  worth  noting.  First,  it  was  assumed  that 
the  sites  interviewed  are  at  least  representative  and  hopefully  ahead  of,  the  larger  AF.  Second,  it 
is  assumed  the  target  audience,  WPC,  has  fundamental  knowledge  of  air  campaign  planning  and 
terminology.  This  audience  includes  analysts,  wargamers,  trainers,  and  warfighters.  Third,  it  is 
assumed  that  WPC  desires  the  ability  to  perform  air  campaign  analysis  autonomously,  even 
though  the  evolving  Expeditionary  Aerospace  Force  (EAF)  concept  may  include  centralized, 
independent  planning  and  analysis. 
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The  background  information  and  statement  of  the  problem,  research  methodology,  and  the 
brief  description  of  limitations  and  assumptions  lays  the  foundation  for  this  research.  The 
findings  will  be  presented  in  the  following  chapter. 


Notes 

*  Briefing,  WPC,  subject:  Nimble  Lion  Initial  After  Action  Review,  August  1998. 

2 

Email  Message,  FW:  Update  (Incorrect  Email)  to  “WPC  Assignment/ ACSC  Research 
Topics”  Email,  Mr.  Jeff  Bradshaw,  26  August  1998. 

Briefing,  WPC,  subject:  WPC  Overview,  no  date. 

^  C.  William  Emory,  Business  Research  Methods  (Richard  D.  Irwin,  Inc. 1985),  I60-I69. 
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Chapter  2 


Findings 


There  is  the  wealth  of  knowledge  that  exists  in  the  Air  Force  (AF)  today.  By  visiting  the 
key  AF  sites  currently  engaged  in  activities  similar  to  the  WPC,  it  is  possible  to  take  a  broad  and 
objective  look  at  the  analysis  processes  and  associated  analysis  tools.  The  knowledge  gained  by 
visiting  the  sites  through  interviews  and  email  is  voluminous.  In  order  to  provide  a  coherent 
overview  of  this  information,  findings  will  be  broken  into  two  major  areas:  Site  Summaries,  and 
Description  of  Analysis  Tools. 


Site  Summaries 

A  description  of  the  AF  sites  visited  is  relevant  to  this  research  since  the  intent  is  to  find 
insights  about  their  processes  and  tools  and  apply  them  to  WPC.  Toward  this  end,  a  brief 
summary  describing  each  site,  its  mission  and  activities,  a  list  of  its  analysis  tools,  and  their 
processes  is  provided  below.  A  description  of  the  analysis  tools  listed  in  the  site  summaries  will 
be  provided  in  the  subsequent  section. 

WPC^ 

The  WPC  is  located  at  Einsiedlerhof  AS,  Germany.  It  is  a  joint  AF/Army  organization  that 
works  for  United  States  Air  Force  Europe  Deputy  of  Operations  (USAEE/DO)  and  US  Army 
Europe  Deputy  Chief  of  Staff  Operations  (USAREUR/DCS  OPS).  The  current  WPC  mission  is: 
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The  Warrior  Preparation  Center  provides  commanders  and  their  battlestaffs,  a 
realistic  joint  and  combined,  operational  level  training  and  exercising 
environment  at  any  location  in  the  theater  or  worldwide  to  maintain  mission 
readiness.^ 

WPC  activities  include  :  designing,  conducting,  and  analyzing  Computer  Assisted  Training 
Exercises  (CPX),  conducting  mission  rehearsals,  and  providing  planning  and  technical  support 
expertise.  WPC  tools  include  an  entire  suite  of  training/exercise  tools  called  Joint  Training 
Confederation  (JTC).  The  Air  Warfare  Simulation  (AWSIM)  is  the  centerpiece  of  this  suite  and 
will  be  discussed  in  more  detail  in  the  next  section. 

The  WPC  has  several  processes  associated  with  these  activities.  However,  since  this 
research  focuses  on  the  mission  rehearsal  activity,  WPC’s  recently  documented  “Mission 
Rehearsal”  process  is  depicted  in  figure  1.  Notice  that  the  specific  steps  start  with  the  ATO  and 
Air  Coordination  Order  (ACO)  creation  followed  by  model  execution  and  concluding  with  a 
briefing.  The  four  major  phases  assigned  to  the  12  steps  are  highlighted  in  figure  1.  They  are 
Preparation,  Execution,  Analysis,  and  Production.  It  should  be  noted  that  this  process  is  used 
primarily  for  training  (JEC  staff  level)  and  more  recently  has  been  expanded  to  include  mission 
rehearsal  for  real-world  contingencies. 


Figure  1.  WPC’s  12  Step  Mission  Rehearsal  Process" 
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Checkmate^ 

Checkmate  is  located  at  the  Pentagon,  Washington  D.C.  It  is  an  AF  organization  that  works 
directly  for  AF/XO.  It  works  closely  with  the  Air  Force  Studies  and  Analysis  (AFSAA) 
organization.  The  current  Checkmate  mission  is  to  provide  operational  assessments  as  directed 
by  the  AF/XO,  CSAF  and  Unified  CINCs.  Typical  Checkmate  activities  include  doctrine  and 
tactics  assessments,  air  campaign  development,  assessments  and  analysis,  and  development  of  air 
campaign  decision  and  planning  tools.  Current  air  campaign  analysis  tools  include:^ 
THUNDER,  Combat  Forces  Assessment  Model  (CFAM),  Operations  Planner  (OPS  Planner), 
Extended  Air  Defense  Simulator  (EADSIM),  and  ORCA  Planning  and  Utility  System  (OPUS). 

Checkmate  analytic  processes  focus  at  the  multiple  ATO-level.  They  typically  don’t 
analyze  individual  ATOs,  although  the  end  result  of  their  analyses  can  be  text  formatted  ATOs. 
Their  integrated  “suite”  of  EADSIM-centered  analysis  tools  was  quite  impressive.  The  tools 
were  designed  to  be  interoperable  and  lend  themselves  to  a  more  automated  and  integrated 
analysis  process.  Einally,  they  rarely  perform  rigorous  campaign-level  analyses  but  do  so 
occasionally  in  association  with  AESAA.  They  are  somewhat  unique  in  that  they  do  more 
qualitative  analyses  relying  on  their  exceptional  warrior-staff  expertise.  These  analyses  focus 
more  on  objective/strategy  congruence  and  rely  on  their  hand-selected  personnel. 

C2TIC’ 

The  C2TIC  is  primarily  located  at  Hurlburt  Eield,  Elorida  with  several  CONUS 
detachments.  It  is  an  AE  organization  that  works  directly  for  the  Aerospace  Command  and 
Control,  Intelligence,  Surveillance,  and  Reconnaissance  Center  (AC2ISRC).  As  the  only 
AC2ISRC  field  unit,  the  C2TIC  mission  is  to  provide  leadership  for  AE-wide  Command  and 
Control.  Typical  C2TIC  activities  include  operational  assessments  of  C2  systems;  C2  system 
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testing;  JFACC-level  training;  conducting  training,  operational  exercises,  and  experiments;  and 
baselining/managing  Air  Operations  Center  (AOC)  systems,  processes,  and  training.  Current  air 
campaign  analysis  tools  include:  Theater  Battle  Management  Core  Systems  (TBMCS), 
Contingency  Theater  Automated  Planning  System  (CTAPS),  the  JFACC  Planning  Tool  (JPT), 
and  AWSIM.  The  last  two  tools  will  be  described  in  the  next  section. 

The  C2TIC  processes  focus  on  training  of  JFACC  staff  personnel;  and  since  they’ve  been 
chartered  to  standardize  AOC  (systems,  processes,  and  training)  -  they  closely  emulate  the 
processes  that  are  in  use  or  that  ideally  could  be  used  by  the  Numbered  Air  Forces  (NAFs). 
Their  training  arm,  the  Command  and  Control  Warrior  School  (C2WS),  formerly  the  Battlestaff 
Training  School  (BTS),  includes  a  classroom  portion.  The  process  depicted  in  figure  2  is  a 
simplified  picture  of  air  campaign  planning  process. 


PLANNING  PPOCLSS 
-  A  SIMPLIFILD  yiLW 


Figure  2.  C2WS’  Simplified  Air  Campaign  Planning  Process* 

More  specific  C2TIC  activities,  including  analysis  tools,  used  for  training  and  exercises,  are 
depicted  in  figure  3.  Although  it  doesn’t  depict  creation  and  execution  of  the  ATO,  these 
activities  do  occur  via  the  CTAPS  network.  So,  in  essence  the  C2TIC  is  executing  the  ATO  like 
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WPC  and  also  performing  higher  level  analyses  like  Checkmate.  Notice  the  Master  Air  Attack 
Plan  (MAAP)  as  an  intermediate  step  between  the  JFACC  staff  and  the  actual  ATO.  The  MAAP 
essentially  pairs  specific  weapons  against  specific  targets.  This  step  is  critical  to  developing  a 
detailed  ATO,  and  many  trade-offs  are  made  and  analyzed  in  the  process  of  narrowing  broad 
resources  and  target  priorities  down  to  weapon/target  pairing.  Finally,  it  should  be  noted, 
however,  that  the  C2TIC’s  ATO  execution  is  geared  toward  training  and  experimentation  and  not 
meant  to  assess  real-world  air  campaigns. 


JFACC  Planning  Tool 

R&h 

High  Level  Guidance 

•  National  Political/Military  Objectives 

•  Theater  Objectives 

•  JFC’s  Guidance 


.  / 


CTAPS  Network 


Figure  3.  JFACC  Planning  Process’ 


AFWI^“ 

The  AFWI  is  located  at  Maxwell  AFB,  Alabama.  It  is  an  AF  organization  that  works 
directly  for  the  Air  University  (AU).  The  current  AFWI  mission  is  to  provide  wargaming  and 
education  to  the  AF,  primarily  the  Professional  Military  Education  (PME)  organizations. 
Typical  AEWI  activities  include  conducting  wargames  for  education  purposes,  supporting  other 
service  wargames  and  using,  assessing,  and  developing  modeling  &  simulation  (M&S)  and 
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analysis  tools.  Current  air  campaign  analysis  tools  include  the  ACT  ATO  Generator  (AAG)  - 
described  in  the  next  section,  THUNDER,  TACWAR,  and  EADSIM. 

The  AEWI  processes  are  mainly  aimed  toward  education  purposes.  They  do  however 
support  major  exercises  like  Prairie  Warrior  and  create  an  education  environment.  This  process 
takes  students  from  NCA  level  objectives  down  to  creation  of  ATO  level  -  thus  encompassing 
the  entire  air  campaign  planning  process.  Although  they  have  air  campaign  analysis  tools  such 
as  THUNDER  and  EADSIM,  they  have  developed  AAG  specifically  for  this  purpose.  It  enables 
students  to  execute  an  entire  air  campaign  and  see  the  results  of  their  decisions.  This  is  unique 
from  the  other  sites,  which  have  several  tools  that  span  only  portions  of  the  air  campaign 
planning  process. 

AFAMS^^ 

Although  AEAMS  is  not  an  “end-user”  of  air  campaign  analysis  tools,  they  work  closely 
with  all  the  organizations  listed  above.  They  were  added  as  a  site  for  this  research  because  of  an 
on-going  study  of  Decision  Support  Tools  for  the  Warfighter  -  closely  paralleling  this  research. 
The  AEAMS  is  located  in  Orlando,  Elorida.  It  is  an  AE  organization  that  works  for  the  Director 
of  Command  and  Control  (AE/XOC).  The  current  AEAMS  mission  is  to  implement 
AE/Joint/DOD  M&S  policy  and  standards,  coordinate  and  manage  major  M&S  programs  and 
initiatives,  support  corporate  AE  M&S  operations,  and  promote  M&S  technology  improvement 
and  innovation.  Since  AEAMS  is  not  an  “end  user,”  typical  activities  and  a  list  of  analysis  tools 
are  omitted.  However,  examination  of  the  on-going  study,  includes  relevant  information 
regarding  the  subject  with  specific  information  about  the  air  campaign  analysis  process  and 
many  analysis  tools. 
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Although  not  an  “end  user”,  AFAMS  has  been  chartered  to  develop  a  roadmap  to  provide 
Warfighters  decision  support  tools.  A  potential  set  of  decision  support  tools  and  the  associated 
process  are  depicted  in  figure  4. 


Roadmap:  ATO  Decision  Support 

Potential  Use 


Figure  4.  Road  Map:  ATO  Decision  Support  Process^^ 

Notice  that  this  process  starts  with  either  a  Course  of  Action  (COA)  or  an  ATO.  Using  the  first 
process,  the  COA  is  converted  into  a  MAAP  using  Ops  Planner,  and  ends  with  a  visualization. 
The  second  process  starts  with  an  ATO,  which  is  then  route-planned  using  OPUS,  and  ends  with 
a  simulation.  Realize  the  intended  users  of  the  above  process  are  real-world  JFACC  staff  and 
more  specifically  AOC  personnel. 


Description  of  Analysis  Tools 

The  amount  of  data  available  for  the  analysis  tools  is  also  enormous;  so  only  a  brief 
summary  of  relevant  information  for  each  tool  will  be  provided.  At  a  minimum,  this  summary 
will  include  the  tool’s  primary  user  and  OPR,  its  operating  environment,  the  tool’s  main  purpose, 
typical  inputs/outputs,  some  qualitative  observations,  and  known  future  plans/transitions.  The 
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presentation  sequence  of  the  tools  is  not  arbitrary.  Their  sequencing  will  become  clear  in  the 
next  chapter. 

AAG^^ 

The  ACT  ATO  Generator  (AAG)  is  used  exclusively  by  AFWI,  who  is  also  the  tools  OPR. 
It  is  PC-based  and  its  primary  purpose  is  to  facilitate  education  of  the  entire  air  campaign 
planning  process.  Typical  inputs  include  broad  objectives  initially,  with  weapons  and  targets  as 
intermediate  inputs.  Intermediate  outputs  include  text- formatted  ATOs  and  actual  execution  of 
the  ATO  followed  by  final  outputs  in  statistical  and  tabular  forms.  Output  information  includes 
aircraft  losses,  history  trends  and  overall  combat  effectiveness.  The  tool  has  inherently  analytic 
properties  but  limited  excursion  flexibility.  The  tool  uses  unclassified  data  and  algorithms;  thus 
isn’t  valid  for  analyzing  real-world  air  campaigns.  It  is  very  timely  to  operate,  modeling  a  30 
day  air  campaign  in  only  a  few  hours,  and  has  a  Graphical  User  Interface  (GUI),  making  it 
relatively  user-friendly.  Some  degree  of  expertise  is  needed  to  use  AAG  and  although  it  does 
“fly-out”  the  ATO,  it  doesn’t  visualize  it.  There  is  no  replacement  in  the  known  future,  and 
being  a  new  tool  it  will  likely  mature  over  time. 

THUNDER^'' 

THUNDER  is  used  by  Checkmate  (via  AFSAA  -  who  is  also  the  tool’s  OPR)  and  AFWI, 
but  is  widely  used  throughout  DOD  analysis  shops.  It  runs  best  on  fast  SUN  workstations  using 
a  Solaris  operating  system.  Its  principal  purpose  is  to  examine  air  and  space  power  effectiveness 
in  a  theater- level  joint  warfare  environment.  It  can  be  run  in  either  analytical  or  wargame  modes. 
In  its  analytical  mode,  attrition  rates;  readiness;  sustainability;  force  structures;  alternative 
courses  of  action  (AC A);  and  evolving  capabilities  and  operational  strategies  can  be  examined. 
In  its  wargaming  mode,  it  provides  a  rudimentary  visualization  (FFOT  movement,  etc)  and  is 


13 


used  to  facilitate  senior  staff  education  and  training.  Inputs  are  voluminous  and  typically  include 
friendly/enemy  joint  forces,  geographic  and  weather  data,  logistics,  etc.  Since  there  is  no  GUI, 
data  input  is  tedious  taking  literally  months  to  prepare  a  single  theater.  However,  once  the 
databases  are  populated,  excursions  and  large  numbers  of  runs  can  be  accomplished  in  a  timely 
manner.  Outputs  are  user  selectable  and  are  in  text  file/tabular  formats.  Typically  outputs 
include  campaign  outcome  information,  equipment  inventories,  planned,  executed,  and  cancelled 
sorties,  etc.  The  tool  is  validated  and  has  a  wide  user  base  but  requires  a  fairly  high-level  of 
personnel  expertise  to  operate.  The  planned  replacement  for  THUNDER  is  the  Synthetic  Theater 
Operations  Research  Model  (STORM)  which  will  operate  in  conjunction  with  the  National  Air 
and  Space  Model  (NASM)  and  will  interface  with  Joint  Warfare  Simulation  (JWARS). 

CFAM^® 

CFAM  was  created  for  AFSAA  who  is  the  tool  OPR.  It  was  created  primarily  as  a  quick, 
albeit  less  powerful,  alternative  to  THUNDER.  Checkmate,  AFSAA,  ACC/SAS,  and  OAS  are 
primary  users  of  CFAM.  It  is  PC  based  and  relies  on  a  fast  linear  program  to  optimize  aircraft- 
weapons-target  pairings.  Its  primary  purpose  is  to  analyze  operational  tradeoffs,  with  the  goal  of 
optimizing  one  (of  five)  air  campaign  objectives,  like  minimizing  attrition,  for  example.  Typical 
inputs  include  weather,  attrition  limits,  budget  and  cost  data,  aircraft  and  target  data,  etc. 
Although  it  requires  a  high  level  of  input  data  initially,  it  does  have  its  own  GUI  and  does 
interface  with  several  already  existing  models.  Outputs  are  text  files  that  can  be  manipulated 
using  analytic  tools  such  as  Excel,  Access,  and  Statistical  Analysis  Software  (SAS).  CFAM  can 
be  used  to  quickly  explore  large  areas  of  interest.  Critical  points,  i.e.  the  ‘knee’  of  the  curve,  can 
then  be  explored  in  detail  with  THUNDER.  Future  enhancements  anticipated  include  a  “back¬ 
end”  GUI  to  increase  utility  and  an  interface  to  OPUS. 
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OPS  Planner^® 

Ops  Planner  is  primarily  used  by  Checkmate,  who  is  the  tool  OPR,  as  a  replacement  to  JPT. 
It  is  PC  based  and  its  purpose  is  to  facilitate  the  strategy  to  task  process.  Typical  inputs  include 
theater-level  objectives,  tasks,  and  measures  of  merit  and  targets.  Output  is  a  text  MAAP,  but 
the  model’s  unique  ability  is  to  look  at  critical  linkages  between  the  tasks  and  the  final  air 
campaign  plan.  The  output  MAAP  is  then  input  into  OPUS  for  routing  and  EADSIM  for  high 
fidelity  attrition  analysis  and  combat  rehearsal.  This  is  a  relatively  new  tool  currently  in  beta 
testing;  its  scheduled  Initial  Operational  Capability  (IOC)  date  is  April  1999. 

JPT^’ 

JPT  was  bom  from  Checkmate’s  Air  Campaign  Planning  Tool  (ACPT)  and  is  intended  for 
AOC  staff  use.  Of  the  sites  visited,  the  C2TIC  is  the  primary  user.  It  currently  mns  as  part  of 
CTAPS  and  is  primarily  used  to  facilitate  weapons/target  pairing.  Typical  inputs  include 
Commander’s  guidance;  objectives  and  tasks;  Intel  analysis;  targets;  and  aircraft/munitions 
available.  Some  of  this  data  must  be  inputted  manually.  Primary  outputs  include  a  prioritized 
list  of  air  tasks,  a  candidate  list  of  targets,  and  weapons/target  pairing.  Some  outputs  are  in  AOC 
standard  formats,  like  Candidate  Target  Lists  and  MAAPS.  JPT  performs  its  functions  well, 
given  an  “air  smart”  operator.  However,  it  is  not  user-friendly  enough  to  be  widely  accepted  by 
AOC  staffs  and  is  not  a  good  tool  for  multiple  excursion  analysis.  Hopefully,  as  JPT  transitions 
to  the  TBMCS  and  as  AF-wide  AOC  standardization  continues,  JPT  will  mature  into  the  tool  it 
was  originally  intended  to  be. 

AMASS^* 

AMASS  is  still  being  developed,  but  is  intended  to  be  used  by  AOC  personnel.  ESC  is 
currently  the  tool’s  OPR.  AMASS  is  TBMCS-based,  primarily  using  an  SGI  environment. 


15 


however,  some  components  have  been  ported  to  Windows  NT  and  Sun  environments.  Its 
purpose  is  to  parse  ATOs  and  provide  some  rudimentary  simulation  capability  of  the  ATO  to 
perform  physics  checks  and  error  checking.  The  later  capability  currently  uses  EADSIM  as  its 
simulation  engine.  Typical  inputs  include  EADSIM  characteristics  data;  some  of  which  can  be 
gleaned  from  ATO  and  AGO  messages.  The  ultimate  intent  is  to  get  all  input  data  from  existing 
TBMCS  or  EADSIM  databases.  Outputs  include  typical  EADSIM  outputs  for  the  simulation 
component  and  ATO  error  log  information.  AMASS  is  still  being  developed  and  thus  it  is  too 
early  to  make  qualitative  assessments. 

EADSIM^^ 

EADSIM  is  widely  used  throughout  the  DOD,  to  include  both  Checkmate  and  AEWI.  It 
runs  primarily  using  an  SGI  environment,  but  can  be  run  using  a  SUN  environment.  EADSIM  is 
an  analytic  model  used  for  a  wide  range  of  applications  focusing  on  air  and  missile  warfare.  It  is 
unique  in  that  it  models  individual  platforms  and  models  interactions  between  C2  processes  and 
intelligence  activities.  Checkmate  uses  it  primarily  to  conduct  attrition  analyses  of  alternative 
war  plans.  Their  inputs  include  loading  enemy  Integrated  Air  Defense  System  (IADS)  primarily 
and  an  already  developed  MAAP  with  large-scale  aircraft  routes  generated  from  OPUS.  Data 
input  is  somewhat  of  an  art.  However,  given  semi-processed  data,  i.e.  from  OPUS,  or 
warfighter-interpreted  data,  a  single  ATO  can  be  entered  in  a  day.  Eaying  down  an  Iraqi  size 
IADS  from  scratch,  however,  takes  several  weeks.  Outputs  are  user  selectable,  but  typically 
include  attrition  data  and  a  visualized  fly-out  of  the  entire  ATO  (or  MAAP/OPUS  Routes).  Once 
the  database  for  an  AOR  is  entered,  excursions  are  relatively  easy;  for  example  a  Desert  Pox  type 
operation  was  fully  analyzed  in  2-3  days  by  2-3  experts.  The  model  is  validated  and  interfaces 
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well  with  several  Checkmate  created  (or  enhanced)  tools.  A  PC  version  of  EADSIM  may  be 
available  in  a  few  years. 

OPUS^® 

OPUS  is  primarily  used  by  Checkmate.  It  is  PC  based  and  its  purpose  is  to  autoroute  large 
numbers  of  aircraft  through  enemy  IADS.  The  primary  input  is  a  MAAP.  Outputs  include 
aircraft  routing  (for  EADSIM)  and  the  “fly-out”  can  be  visualized.  In  the  future  it  will  get 
imports  from  CEAM  and  OPS  Planner.  Thus  OPUS,  plus  CEAM  and  EADSIM  offer  an 
integrated  “suite”  of  tools  that  collectively  form  a  formidable  analysis  tool  set.  OPUS  should  not 
be  considered  on  a  stand-alone-basis,  unless  only  autorouting  large  numbers  of  aircraft  through 
enemy  IADS  is  needed. 

AWSIM^^ 

AWSIM  is  primarily  used  by  the  C2TIC  and  WPC.  ESC  is  the  tool  OPR,  although  the 
AEAMS  controls  the  configuration.  It  runs  using  a  SUN  environment.  The  primary  purpose  is 
modeling  the  ATO  execution  for  training,  and  less  so,  for  wargaming  purposes.  Inputs  include 
the  ATO,  directly  from  CTAPS,  and  enemy  threat  data.  Outputs  include  visualization  of  the 
ATO  “fly-out”  and  other  information  for  training  critiques.  AWSIM  interfaces  with  many 
models  including  ground  and  naval  models.  Thus  to  really  reap  the  full  utility  of  AWSIM,  the 
AWSIM  “suite”  of  models,  or  the  JTC,  should  be  used.  AWSIM  executes  ATOs  and  interfaces 
with  other  models  extremely  well,  however,  it  was  not  designed  for  analytical  purposes;  and  has 
very  limited  excursion  flexibility.  The  model  has  been  calibrated  for  training  and  has  a  stable 
user  base.  Once  the  databases  for  an  AOR  are  populated  it  can  be  run  faster  than  real  time. 
Unlike  most  of  the  other  models,  there  is  considerable  government  expertise  in  operating 


17 


AWSIM.  The  future  of  AWSIM  is  limited  since  it’s  scheduled  to  be  replaced  by  Joint 
Simulation  (JSIMS)  in  the  next  few  years. 

A  lot  of  information  concerning  air  campaign  analysis  activities  was  presented.  Dividing 
the  information  into  site  summaries  and  tool  descriptions  was  done  in  order  to  make  the 
information  more  understandable.  Based  on  this  information,  some  meaningful  analysis  can  be 
performed  and  is  introduced  in  the  following  chapter. 


Notes 

*  Mr.  Edward  Ballanco  (et  al),  WPC,  interviewed  by  author,  15  December  1998. 
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^  Major  Douglas  Fuller,  Checkmate,  interviewed  by  author,  29  December  1998. 

^  Briefing,  Checkmate,  Subject:  Checkmate  Analysis  Tools,  22  July  1998. 

^  Mr.  Don  Neal  (et  al),  C2TIC,  interviewed  by  author,  29  January  1999. 
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Briefing,  C2TIC,  Subject:  Joint  Air  Planning,  no  date. 

^  Briefing,  C2TIC,  Subject:  JFACC  Planning  Tool,  no  date. 

Dr.  E.L.  Perry,  AFWI,  interviewed  by  author,  5  February  1999. 

**  AFAMS  Website,  www.afams.af.mil. 

1 2 

Briefing,  AFAMS,  Subject:  Improving  Modeling  and  Simulation  Decision  Support  Tools 
for  the  Warfighter,  Dr.  Flash  Gordon,  24  January,  1999. 

Dr.  E.L.  Perry,  AFWI,  interviewed  by  author,  5  February  1999. 

Email  Message,  FW:  THUNDER  Info,  Mr.  Roger  Weissflog,  4  February  1999. 
THUNDER  Website,  available  from  http://www.s3i.com/tug_home.htm. 

Briefing,  Checkmate,  Subject:  Checkmate  Analysis  Tools,  22  July  1998. 

Major  Douglas  Fuller,  Checkmate,  interviewed  by  author,  29  December  1998. 

“CFAM,  Combat  Forces  Assessment  Model,  Training  CD”,  CD-ROM,  ASI  Systems 
International,  31  July  1998. 

Major  Douglas  Fuller,  Checkmate,  interviewed  by  author,  29  December  1998. 

Mr.  Clay  Olschner,  C2TIC,  interviewed  by  author,  29  January  1999. 

Email  Message,  RE:  AMASS  Info,  Mr.  Jonathan  Prescott,  19  February  1999. 

Facsimile  Message,  no  subject,  Mr.  Jim  Hingst,  25  January  1999. 

Email  Message,  RE:  EADSIM  Overview,  Mr.  Jim  Hingst,  28  January  1999. 

Briefing,  Checkmate,  Subject:  Checkmate  Analysis  Tools,  22  July  1998. 

Email  Message,  Subject:  AWSIM  User’s  Manual,  Ft.  Joseph  Hernandez,  29  January 

1999. 

AWSIM  Website,  available  from  http://www.wg.hanscom.af.mil/AWSIMR/. 
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Chapter  3 


Analysis 


The  findings  showed  some  useful  process  insights  and  potentially  promising  tools  for 
WPC’s  use.  It  is  important  to  synthesize  these  findings  into  a  coherent  and  meaningful  way.  To 
do  so,  this  chapter  will  first  look  at  the  various  processes  in  use  by  the  sites  and  develop  a 
notional  air  campaign  analysis  process.  Second,  the  individual  tools  will  be  assessed  based  on 
common  factors  deemed  important  by  the  various  tool  users.  Third,  the  two  analyses  will  be  fit 
together  into  a  coupled  process  and  then  applied  to  a  WPC-specific  example. 

Analysis  of  Processes 

Several  trends  are  observable  based  on  the  sites’  activities  and  the  specific  functions  of  their 
analysis  tools.  Ideally,  air  campaign  analysis  occurs  during  and  throughout  the  air  campaign 
planning  process.  Using  the  air  campaign  planning  process  depicted  in  figure  2  as  a  basic 
framework,  and  the  sites’  activities,  tools,  and  processes,  two  major  trends  emerge.  First,  there  is 
an  increasing  level  of  specific  knowledge  about  the  air  operations;  going  from  broad  objectives 
to  a  very  specific  ATO.  Second,  there  is  a  sequentially  narrowing  of  alternatives  from  multiple 
courses  of  action  to  a  single  air  campaign  plan.  These  two  trends  provide  the  two  major  axes  for 
a  notional  air  campaign  analysis  process,  depicted  in  figure  5.  This  first  major  trend  will  be 
defined  as  “Air  Operations  Specificity,”  which  increases  as  the  process  advances.  The  second 
major  trend  will  be  defined  as  “Analysis  Flexibility,”  which  decreases  as  the  process  advances. 
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Having  framed  the  continuum  of  the  air  campaign  analysis  process,  discrete  subprocesses 
need  to  be  defined  to  make  the  process  practical.  Discrete  subprocesses  will  be  defined  with 
three  key  factors  in  mind:  the  functions  performed,  the  tools  used,  and  the  organization 
responsible  to  perform  the  function.  The  predominance  of  activities,  analysis  tools,  and 
organizational  missions  center  around  three  analysis  levels.  The  lowest  level,  analyzing  the 
effectiveness  of  individual  aircraft  missions  against  specific  targets  will  be  defined  as  “ATO- 
Level  Analysis.”  The  next  higher  level,  evaluating  the  effectiveness  of  alternative  target  sets 
and  weapons  and/or  specific  aircraft  type  to  target  type  (or  in  some  cases  an  actual  MAAP)  will 
be  defined  as  “MAAP-Level  Analysis.”  Finally,  the  highest  level  of  analysis,  starting  with  joint 
objectives  and  comparing  multiple  COAs  against  those  objectives  will  be  defined  as 
“Campaign-Level  Analysis.”  Naturally,  there  can  be  overlaps  between  the  levels,  degrees  of 
independence,  and  two  way  iterations  between  them.  However,  for  a  simplified 
conceptualization  of  the  entire  process,  the  three  discrete  levels  defined  above  capture  the  entire 
spectrum  of  the  air  campaign  planning  analysis.  Also  segregation  of  the  levels  lends  itself  to 
organizational  separation  of  analyses  and  more  manageable  subprocesses. 


Figure  5.  Notional  Air  Campaign  Analysis  Process 


20 


Notice  in  figure  5  the  basic  framework  of  this  notional  process  rests  on  the  two  axes  defined 
earlier:  Analysis  Flexibility,  which  decreases  over  time,  and  Air  Operations  Specificity,  which 
increases  over  time.  Next,  notice  each  of  the  three  analysis  levels.  Campaign,  MAAP,  and  ATO, 
has  individual  inputs  and  outputs,  thus  can  be  performed  independently.  Also  notice,  the  three 
levels  are  arranged  sequentially  and  include  arrows  between  them,  symbolizing  the  potential 
interlinkage  of  each  higher  level  analysis  feeding  the  next  lower  level.  This  notional  process  will 
be  more  applicable  once  specific  inputs/outputs  for  each  of  the  three  analysis  levels  and  specific 
analysis  tools  are  identified.  But  before  doing  so,  an  analysis  of  the  tools  described  in  the 
previous  chapter  must  be  accomplished. 

Analysis  of  Tools 

Given  the  concept  of  three  distinct,  but  interrelated  and  overlapping  analysis  levels,  the  tools 
observed  can  be  more  appropriately  assessed.  No  one  tool,  with  the  exception  of  AFWI’s  AAG, 
can  meet  the  needs  of  the  entire  spectrum  of  air  campaign  analysis.  However,  based  on 
interviews  with  the  various  sites  and  their  selection,  creation,  and  refinement  of  the  tools  they 
use,  some  common  criteria  are  apparent.  These  criteria  are  qualitative  in  nature,  and  thus  a 
quantitative  assessment  or  comparison  of  the  various  tools  is  unfeasible.  Nonetheless,  these 
criteria  are  relevant  and,  at  a  minimum,  invoke  a  meaningful  thought  process  when  assessing  the 
tools  for  consideration.  Common  criteria  for  assessing  the  utility  of  the  tools  include: 

•  Powerfulness  and  Breadth  of  Tool  Application 

•  Experiment  Design  and  Excursion  Elexibility 

•  Tool  User  Base  (there’s  safety  and  efficiency  in  numbers) 

•  Input  Database  Compatibility/Ease  of  Data  Entry 

•  Output  Data  Compatibility  (to  next  process  input)/Eormat  Specific  Outputs 

•  Tool  Interfaces  to  Other  Tools  and  Models 

•  Timeliness  in  Using  the  Tool  (How  long  to  set  up,  run,  and  iterate) 

•  User  Eriendliness  of  the  Tool/Personnel  Expertise  Required 

•  Modeling  Ely-Out  and  Visualization 
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The  relative  importance  of  each  criterion  varies  by  individual  organizational  tastes  and  needs. 
But,  a  common  perceived  trend  is  that  the  importance  of  these  criteria  correlates  with  the  air 
campaign  analysis  level.  For  example,  for  Campaign-Level  Analysis,  the  tool’s  breadth  and 
excursion  flexibility  are  critical;  but  the  need  to  fly-out  and  visualize  individual  aircraft  is  much 
less  important.  At  the  other  end  of  the  air  campaign  analysis  spectrum  conversely,  fly-out  and 
visualization  are  paramount  while  the  tools'  ability  to  perform  broad  analysis  functions  isn’t. 
The  notional  analysis  sublevels.  Campaign,  MAAP,  and  ATO  and  the  tools  reviewed  are 
interrelated.  To  meaningfully  compare  the  tools,  they  must  first  be  put  into  the  framework  of  the 
notional  air  campaign  analysis  process.  Further,  the  tools  need  to  be  grouped  into  the  three 
analysis  levels.  However,  just  as  the  individual  analysis  levels  overlap,  so  do  the  tools.  A 
graphical  depiction  of  the  tools  grouped  by  analysis  level  is  provided  in  figure  6. 


TOOLS  APPLICATION  CONTINUUM 


Figure  6.  Tools  Application  Continuum 
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In  figure  6,  notice  the  tools  cascade  from  left  to  right  starting  with  Campaign-Level  tools  toward 
ATO-Level  tools.  Also  notice  that  no  tool  fits  exclusively  into  one  analysis  level;  thus 
emphasizing  the  overlap  between  the  three  analysis  levels  themselves.  Discussing  the  relative 
utility  of  each  individual  tool  can  now  be  meaningfully  done  using  both  the  tool  criteria 
described  in  the  previous  section  and  the  three  groupings  depicted  in  figure  6. 

The  tools  will  be  compared  within  the  three  analysis  levels.  However,  since  AFWI’s 
AAG  extends  to  all  levels  of  analysis  it  will  be  discussed  separately.  Using  a  tool  that  performs 
analyses  across  the  entire  spectrum  seems  ideal.  However,  AAG  is  used  predominantly  for 
education  purposes;  and  not  real-world  operations.  Also,  since  it  uses  unclassified  algorithms 
and  databases,  significant  modifications  are  required  for  it  to  be  fully  useful  to  WPG.  However, 
the  ability  to  go  from  high-level  campaign  planning  to  individual  ATO  fly-out  and  see  the  results 
of  planners  efforts  from  start  to  finish  is  unparalleled. 

Campaign-Level  Analysis  tools  include  THUNDER,  CFAM,  and  OPS  Planner. 
THUNDER  is  predominantly  a  campaign  level  analysis  tool  and  has  little  overlap  into  the 
MAAP  level.  Of  the  three,  it  is  the  most  powerful,  has  the  widest  breadth  of  use,  and  is 
conducive  to  experimental  designs  with  high  volumes  of  excursions.  It  is,  however,  difficult  to 
use  and  requires  high  levels  of  expertise  to  operate  and  is  best  used  by  skilled  analysts.  CFAM, 
conversely  is  PC-based,  easier  to  use,  but  less  powerful  than  THUNDER.  However,  it  can 
quickly  explore  large  areas  of  interest,  finding  the  ‘knee’  of  the  curve.  It  does  overlap  into  the 
MAAP  level  of  analysis  and  will  eventually  provide  outputs  compatible  with  OPUS,  and 
indirectly,  EADSIM.  OPS  Planner  is  also  PC -based,  but  is  a  relatively  new  tool.  It  promises  to 
replace  JPT  and  its  output  MAAP  can  be  input  directly  into  OPUS. 
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MAAP-Level  Analysis  tools  include  OPS  Planner,  JPT,  AMASS,  and  EADSIM.  OPS 
Planner,  as  previously  described,  is  relatively  new,  but  is  PC-based  and  extends  well  into 
Campaign-Level  Analysis.  The  JPT  is  currently  available  on  CTAPS  and  will  be  available  on 
TBMCS;  both  platforms  are  intended  for  AOC  and  planning  cell  staff  use.  It  has  very  specific 
inputs  and  outputs  and  is  a  good  tool  for  creating  MAAPs  from  task,  targets,  and  resource  lists. 
To  be  fully  embraced  by  its  current  users,  however,  JPT  needs  some  major  refinements  but  may 
mature  over  time.  AMASS,  may  eventually  replace  JPT,  and  is  likewise  intended  for  AOC  and 
planning  cell  staff  use.  It  will  be  incorporated  into  later  versions  of  TBMCS  and  is  envisioned  to 
provide  ATO  error/reality  checks.  Finally,  EADSIM  is  a  workhorse  that  spans  between  high 
level  analyses  down  to  individual  aircraft  modeling  and  visualization.  It  assesses  attrition  rates, 
models  individual  entities  (joint)  and  provides  a  visual  display  at  campaign  levels  down  to  ATO 
levels.  It  is  validated  and  has  a  wide  user  base.  It  is,  however,  difficult  to  populate  and  requires 
a  moderate  level  of  expertise  to  operate.  It  does,  none  the  less,  have  a  user-friendly  interface 
specifically  designed  to  allow  non-  analysts  to  use  it. 

ATO-Level  Analysis  tools  include  EADSIM,  OPUS  and  AWSIM.  EADSIM,  as 
described  previously,  is  a  powerful  tool  extending  well  into  MAAP-Level  Analysis;  but  requires 
a  moderate  level  of  expertise.  It  does,  however,  offer  analytical  capabilities  well  beyond  OPUS 
or  AWSIM,  and  interfaces  to  several  other  tools  (CFAM,  OPS  Planner,  AMASS,  OPUS).  OPUS 
is  really  a  mission-planning  tool  for  large-scale  operations.  It  interfaces  with  EADSIM,  CFAM, 
and  OPS  Planner.  It  does  provide  visualization  and  fly-out,  but  models  friendly  aircraft 
survivability  more  than  combat  effectiveness.  It  has  been  suggested^  that  before  a  meaningful 
simulation  of  an  ATO  can  be  accomplished,  optimized  routing  is  needed  to  provide  a  more 
realistic  assessment  of  friendly  attrition  through  enemy  IADS.  Finally,  AWSIM  is  intended  for 
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training  and  exercise  purposes,  and  has  little  analytical  capability.  However,  WPC,  recognizing 
this,  used  it  for  Nimble  Lion  as  the  best  tool  available  at  the  time.  It  does  provide  some  overlap 
into  MAAP,  but  is  really  best  suited  for  individual  ATO  modeling  and  fly-out.  It  has  the  widest 
user  base  of  all  the  models  investigated  and  interfaces  well  to  other  joint  models;  allowing 
analysis  of  other  than  air  operations. 


Application  of  Process  and  Tools  Analyses 

Using  the  above  tools’  assessment,  specific  tools  and  representative  inputs/outputs  can  be 
added  to  each  analysis  level.  This  expansion  to  the  notional  air  campaign  analysis  process  is 
applicable  to  a  WPC-specific  example  and  is  depicted  in  figure  7. 
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Figure  7.  Applied  Air  Campaign  Analysis  Process 

Notice  in  figure  7  the  two  major  axes.  Air  Operations  Specificity  and  Analysis  Flexibility 
still  frame  the  process.  Also,  the  specific  inputs/outputs  for  each  analysis  level  correspond  to  the 
tools  recommended  and  are  intended  to  be  representative,  but  not  all  encompassing. 
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To  demonstrate  the  application  of  the  notional  air  campaign  analysis  process,  a  hypothetical 
example  will  walk-through  figure  7.  Imagine  WPC  is  tasked  to  assist  USAFE  in  Crisis  Action 
Planning  (CAP)  for  a  real-world  contingency.  First,  the  Campaign-Level  Analysis  begins  with 
CINC-level  joint  objectives,  apportioned  joint  forces,  and  some  information  about  the  AOR  and 
the  enemy  capabilities.  Potentially,  other  information,  like  multinational  forces  available, 
assumptions.  Commander’s  Intent,  a  Mission  Statement,  and  specific  JFACC  and/or  J3/J5 
guidance  is  provided  as  well.  WPC  analysts  gather  experiment  design  information,  especially 
desired  decision  factors  such  as  attrition  rates,  combat  effectiveness,  etc  and  then  determine  the 
types  and  numbers  of  excursions  needed.  This  would  facilitate  populating,  or  at  least  bounding, 
the  solution  volume;  precluding  tedious  and  time-consuming  rework  later.  Using  analysis  tools, 
like  CFAM,  results  in  broad-level  analysis;  identifying  “knee”  in  the  curve  critical  points  of 
interest.  These  activities  segregate  the  airpower  portion  from  the  overall  joint  campaign  plan, 
assess  ACAs,  and  ultimately  result  in  several  candidate  COAs. 

Next,  the  MAAP-Level  Analysis  starts  with  these  candidate  COAs;  which  could  be  provided 
to  WPC  (directly  from  USAFE  or  from  a  centralized  planning  cell).  WPC  analysts  gather 
information  in  addition  to  the  COAs,  such  as  JFACC-prioritized  tasks,  prioritized  target  lists  and 
apportioned  airpower  resources.  At  this  point,  the  desired  decision  factors  and  excursions  are 
narrower,  having  “modeled-out”  undesirable  or  less  feasible  alternatives.  Using  analysis  tools, 
like  the  EADSIM  “suite”  of  tools  accomplishes  assessments  of  specific  weapon/target  pairings, 
comparisons  of  candidate  COAs  and  ultimately  provides  candidate  ATOs.  These  candidate 
ATOs  could  be  either  the  initial  day  of  the  air  campaign  or  could  be  progressive  multiple-day 
ATOs.  The  ATOs  associated  with  each  COA,  or  an  optimum  COA,  is  then  provided  for  the  next 
level  for  analysis. 
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The  ATO-Analysis  Level  starts  with  ATOs  developed  from  the  MAAP-Level  Analysis; 
which  could  be  provided  to  WPG  (directly  from  USAFE  or  from  a  centralized  planning  cell). 
WPG  analysts  gather  information  in  addition  to  the  ATOs,  such  as  specific  AOR  or  the  latest 
enemy  threat  assessment  for  realistic  ATO  modeling.  At  this  point  the  decision  factors  and 
number  of  excursions  should  be  narrowest.  Using  analysis  tools,  like  the  AWSIM  “suite”  of 
tools,  specific  ATOs  are  assessed,  compared,  and  optimized.  Ultimately,  a  final,  optimum  ATO 
is  provided  to  the  warfighter/training  audience. 

This  chapter  synthesized  the  abundant  findings  from  the  various  sites  in  a  coherent  and 
meaningful  way.  This  was  accomplished  by  first  looking  at  a  notional  process  framework  across 
the  entire  spectrum  of  the  air  campaign  analysis  process.  Next,  the  individual  tools’  were 
grouped  into  the  three  analysis  levels  and  then  compared.  Goupling  the  notional  process  and 
evaluated  analysis  tools  resulted  in  an  applied  air  campaign  analysis  process  with  a  WPG- 
specific  example.  The  utility  of  this  analysis  culminates  with  specific  recommendations, 
presented  in  the  next  chapter. 

Notes 

*  Email  Message,  EW:  Decision  Support  Tools  Briefing,  Dr.  Elash  Gordon,  11  Eebruary 
1999. 
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Chapter  4 


Conclusions 

The  information  gathered  from  the  various  Air  Force  sites  and  subsequent  analyses  have 
provided  great  insight  into  improving  the  WPC  air  campaign  analysis  process.  Although  each 
site  has  unique  missions,  there  is  ample  commonality  and  many  great  ideas  to  be  useful.  Based 
on  these  observations,  recommendations  for  improvement,  with  some  implementation 
considerations  can  be  made.  Finally,  since  this  research  was  by  no  means  comprehensive,  there 
are  several  potential  areas  for  future  research  identified. 

Recommendations 

First  and  foremost,  consider  the  Applied  Air  Campaign  Analysis  Process  depicted  in  figure 
7  as  a  framework  capturing  the  entire  spectrum  of  analysis.  Once  the  varying  analysis  levels  are 
segregated,  tools  and  processes  become  more  manageable,  vice  finding  a  single  tool  or  process  to 
do  everything.  WPC  currently  does  ATO-Level  Analysis,  using  AWSIM  primarily,  but  needs 
some  improvement.  However,  instead  of  concentrating  on  improving  this,  adding  the  other 
analysis  levels  would  greatly  expand  WPC  current  capabilities  over  the  entire  air  campaign 
analysis  spectrum.  Although  there  are  several  tools  in  development,  WPC  should  not  wait,  but 
should  instead  expand  its  capabilities  now,  but  do  so  incrementally.  The  incremental  approach 
recommended  below  uses  WPC’s  self-recommended  analytical  improvement  approach  of 
“CRAWL— WALK— RUN.”  ^ 
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CRAWL:  Add  the  ability  to  do  MAAP-Level  Analysis.  AWSIM  is  not  the  right  tool  for 
this,  but  EADSIM  is  well  suited  for  the  task.  Including  the  EADSIM  “suite”  of  tools  provides 
even  greater  capability.  Adding  this  intermediate  step  between  Campaign-Eevel  and  ATO-Eevel 
analysis  would  allow  greater  flexibility  in  a  Nimble  Eion  type  activity.  It  would  provide  broader 
analyses,  with  visualization,  and  help  WPC  to  bound  the  solution  volume  with  more  excursions. 
Most  importantly,  it  would  result  in  narrowing  down  the  ATO  candidate(s)  considerably,  easing 
and  greatly  accelerating  the  ATO-Eevel  analysis,  especially  AWSIM  set-up  and  execution  time. 
Although  EADSIM  “suite”  can  do  some  Campaign-Eevel  analysis,  this  function  could  also  be 
done  by  outside  agencies  or  manually  until  WPC  expands  its  capabilities  to  include  Campaign- 
Eevel  Analysis.  Einally,  EADSIM  extends  into  ATO-Eevel  Analysis  and  provides  analytical 
tools  well  beyond  current  AWSIM  capabilities.  This  CRAWE  step  alone  would  be  a  significant 
improvement  to  the  current  WPC  air  campaign  analysis  process! 

WAEK:  Add  the  capability  to  do  Campaign-Eevel  analysis.  If  WPC  finds  outside  help  in 
this  area  inadequate,  or  wants  greater  self-sufficiency,  this  would  capture  the  full  spectrum  of 
analyses.  Although  this  could  be  done  by  expanding  the  MAAP-Eevel  analysis,  a  more  thorough 
Campaign-Eevel  analysis  capability  is  ideal.  In  order  to  provide  broader  and  more  far-reaching 
analysis,  a  tool,  such  as  CEAM  should  be  considered.  As  before,  there  are  other  tools  in 
development  (STORM,  and/or  JWARS),  but  this  tool  is  available  now  and  used  by  several  AE 
organizations.  Eull  implementation  would  require  higher  levels  of  statistical  analysis  and  greater 
rigor  in  experiment/excursion  design. 

RUN:  Seamlessly  and  interactively  accomplish  all  three  levels  of  analysis.  This  will  take 
experience,  refinements  to  the  WPC  analysis  process  and  greater  interoperability  between  the 
various  tools.  During  the  Crawl  and  Walk  periods,  the  AE  will  continue  to  develop  decision 
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support  tools,  perfect  the  TBMCS/GCCS  tools,  and  continue  to  standardize  AOC  processes  and 
training.  A  growth  path  toward  these  is  optimal,  since  it  will  allow,  at  a  minimum,  compatibility 
with  1)  real-world  products  like  ATOs,  MAAPs,  etc,  2)  tools  like  JPT,  AMASS,  JOPES,  etc,  and 
3)  standardized  AOC  processes.  Also,  during  this  timeframe,  a  transition  from  the  current 
AWSIM  to  the  NASM  and  ultimately  the  all-service  model,  JSIMS,  should  be  planned  and 
implemented  as  WPC’s  ATO-Level  Analysis  capability  evolves.  Ultimately,  this  RUN  phase 
will  allow  WPG  to  train  using  the  same  tools  used  in  the  field;  and  to  the  degree  desired, 
replicate  those  processes. 

By  incrementally  implementing  and  segregating  the  varying  levels  of  air  campaign  analysis, 
adding  the  analysis  tools  and  processes  becomes  more  achievable.  Doing  so,  will  ultimately, 
give  WPG  the  full-spectrum  of  air  campaign  analysis  and  make  Nimble  Lion  type  activities 
easier,  more  timely,  repeatable,  and  ultimately,  more  meaningful. 

Implementation  Considerations 

Implementing  the  recommendations  above  is  an  extensive  effort  but  would  comprehensively 
add  to  WPG’s  current  capabilities.  Detailed  processes  for  each  analysis  level  need  to  be  defined 
and  further  developed. 

Before  implementing,  reflection  about  WPG’s  relationship  with  outside  organizations  may 
be  warranted.  For  example,  the  USAFE  planning  cell  and/or  the  AOG  may  be  better  suited  to  do 
some  or  all  of  these  analysis  functions.  Or  further  from  home,  consider  collaborating  with  other 
organizations,  such  as  the  still  evolving  Air  Expeditionary  Force  planning  cell,  AGG,  AFSAA,  or 
Gheckmate,  for  higher  level  analyses.  In  that  case,  WPG  need  only  to  define  the  relationship  and 
product  formats  to  be  exchanged.  However,  if  WPG  wants  the  capability  to  perform  the  entire 
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spectrum  of  the  air  campaign  analysis  process,  an  integrated  suite  of  tools,  like  Checkmate’s, 
precludes  the  CRAWL  and  WALK  steps  and  jumps  immediately  to  RUN. 

Also,  WPC  should  consider  the  relationship  and  relative  importance  between  its  training 
mission  and  its  mission  rehearsal  mission.  If  the  two  are  inseparable  and/or  of  nearly  equivalent 
importance,  the  analysis  process  should  consider,  and  be  integrated  with,  the  training  process. 
Furthermore,  selection  of  tools  in  use  AF-wide,  or  at  least  by  WPC’s  training  audience,  should 
be  selected  (as  recommended  in  RUN)  to  be  more  fully  integrated  with  training.  However,  if  the 
training  mission  and  mission  rehearsal  mission  are  not  as  interdependent,  more  stand-alone  and 
more  powerful  analytic  tools  should  be  selected,  perfected,  or  created. 

Finally,  realize  the  AF  at  large  continues  to  wrestle  with  this  very  problem.  As  seen  at 
several  AF  sites,  there  seems  to  be  some  “stove -piping”  of  solutions.  Although  the 
recommendations  include  tools  currently  in  use  by  at  least  two  AF  sites,  WPC  should  keep 
abreast  of  the  AF-wide  plan.  Toward  this  end,  WPC  could  get  on  the  current  TBMCS/GCCS 
“train”,  when  convenient,  skipping  the  intermediate  CRAWL  and  WALK  steps. 

Potential  Areas  for  Future  Research 

This  research,  albeit  limited,  answered  the  immediate  question  of  improving  the  current 
WPC  air  campaign  analysis  capability.  However,  in  addition  to  the  implementation 
considerations  above,  there  are  two  potential  areas  for  future  study,  and  may  be  of  use  to  WPC, 
and/or  the  AF  at  large.  First,  the  sites  where  limited  to  AF  sites  currently  performing  activities 
similar  to  WPC’s.  A  more  extensive  survey,  especially  to  the  NAFs  would  likely  find  even  more 
tools,  processes,  and  innovative  ideas.  Second,  a  more  comprehensive  look  at  the  analysis  tools 
described  herein  may  be  warranted.  The  tools  in  this  study  could  be  researched  more 
extensively.  Certainly,  a  more  hands-on  examination  of  the  tools  in  operation  should  be 
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conducted.  Also,  more  data  on  the  tools’  life  cycle  costs  such  as  tool  training,  operating  costs, 
etc  may  be  needed.  Further,  a  deeper  look  at  related  tools  such  as  logistics  (deployment, 
sustainment  rates,  beddown  etc)  or  specific  related  analysis  tools  (statistical  tools,  databases, 
GUIs,  etc)  may  be  helpful.  In  any  case,  the  future  research  areas  suggested  would  not 
substantially  change  these  conclusions,  but  they  are  a  logical  extension  to  this  initial  research. 

The  author  hopes  this  research  paper  helps  WPG  improve  its  air  campaign  analysis 
capabilities.  At  a  minimum,  the  information  gathered  and  subsequent  analyses  have  provided  an 
objective  assessment  of  WPC’s  current  capabilities  and  some  thought-provoking  ideas.  At  best, 
the  research  results  in  a  tangible  product  for  improving  WPC’s  growing  mission.  Ultimately, 
this  newly  characterized  air  campaign  analysis  process  will  provide  warfighters  thoroughly 
planned  air  campaigns  that  are  fully  analyzed  and  rehearsed  well  before  the  wheels  are  in  the 
well. 


Notes 

’  Briefing,  WPG,  subject:  Nimble  Lion  Initial  After  Action  Review,  August  1998. 
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